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ABSTRACT 

Factors affecting the return rate of a mailed 
questionnaire were investigated. A survey was mailed to 196 students 
who had dropped out of home study courses offered by a distance 
education university. The effects of personalizing the cover letter 
and of including anonymous pre-coded personal information on the 
questionnaire were studied in a 2 X 2 factorial design which also 
allowed for an interaction effect. Reminder cards were sent two weeks 
later. All questionnaires contained pre-coded personal data. There 
were four treatment conditions: (1) personal letter and no option to 
remove pre-coded personal data; (2) personal letter and option to 
remove data? (3) formal letter and no option to remove data; and (4) 
formal letter and option to remove data. Thirty-nine percent of the 
questionnaires were returned. Results showed no interactive effect 
between the kind of cover letter sent and whether or not subjects 
were given the option of tearing off the pre-coded personal data. In 
addition, respondents who were given the option to remove the 
pre-coded data did not exercise that option. There was no difference 
in the response rate according to the type of cover letter or whether 
or not pre-coded information could be removed. (GDC) 
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THE COMBINED EFFECT OF PERSONALIZED APPEAL 
AND PRE-CODING OF PERSONAL DATA ON QUESTIONNAIRE RETURNS 



This study investigated the combined effect on return rates of 
personalizing the cover letter and including pre-coded personal information 
on a mail questionnaire. The 2x2 factorial design allowed for the 
testing of an interaction effect as well as for main effects as 
appropriate. In the event, there was no interactive effect between the 
kind of cover letter sent and whether or not subjects were given the option 
of removing the pre-coded data. In addition, neither of the main effects 
was significant. The finding that the inclusion of pre-coded personal data 
on a questionnaire does not impair return rates may be welcome news to 
investigators conducting studies in which they have prior knowledge of 
respondent characteristics. 
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The Combined Effect of Personalized Appeal and 
Pre-Coding of Personal Data on Questionnaire Returns 

Researchers and practitioners have long been aware of the substantial 
threat that response bis? represents to survey studies using mail-out 
questionnaires. As a result , there has been much attention paid to ways and 
means of increasing the rates of response in such studies and to determining 
what factors in survey and instrument design might influence return rates 
and consequently the representativeness of response. The present study 
examines the combined effect on return rates of personalizing the cover 
letter and including pre-coded personal information on the questionnaire. 

Effects of Personalization 

Andreason (1970, p. 273) has noted that many of the techniques used to 
stimulate response rate "... are in fact designed to personalize the 
communication between the researcher and the respondent. The (often unstat- 
ed) assumption is that the more the respondent feels his unique answers to 
be important, the more likely he is to fill in and return the question- 
naire." However, as Berdie and Anderson (1974, p. 54) have pointed out, 
"Personalization may be accomplished in many ways. ..." The most compel- 
ling, and perhaps the most obvious apprcach has been through a covering 
letter to the respondent. 

A covering letter per se is generally regarded to be an essential 
adjunct to the questionnaire - perhaps to a point where a survey instrument 
should be considered to be both the questionnaire and the cover letter (see, 
for example, Berdie and Anderson, 1974, p. 59). As they have observed, cover 
letters are necessary: (1) to inforrc the respondent about the nature of the 
-tucy zrA hew the data will be utilized; (?) to solicit the participation of 
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the respondent by pointing to tangible and intangible benefits of so doing; 
(3) to provide information about the investigator and/or sponsoring 
organization and about why the study is being conducted; and (4) to infona 
respondents about administrative matters such as how to return the 
questionnaire . However , following the sentiment expressed by Andreason , 
many researchers have investigated whether the effectiveness of a covering 
letter in stimulating response rate could be enhanced by personalizing it in 
some way. 

Longworth (195J) reported that a personal note and typed letter in- 
creased returns by some five percentage points - a difference that might be 
regarded as suggestive but which is clearly not conclusive. Linsky (1965) 
systematically varied the "personalization of the cover letter through use 
of a handwritten personal salutation and signature" (p. 183) with three other 
factors in a 2x2x2x2 factorial design. He found that none of the 
interactions were significant, but the main effect due to personalization 
(as defined by him) was significant. The return rate from the sample 
receiving a letter with personal salutation and signature was significantly 
better than the return rate from the sample receiving a letter with a 
mimeographed salutation and signature. 

Simon (1967) conducted a series of three studies investigating the 
effect of sending personally typed versus form letters to elicit responses 
to mail questionnaires. He reported mixed results that he concluded " . . . 
indicate that personally typed cover letters have no clear-cut advantage 
over mimeographed form letters in terms of responses in surveys using mail 
questionnaires" (p. 29). Furthermore, he speculated that a personally typed 
cover letter can inhibit response if the individuals being surveyed feel 
•^rtir*:l arly tr ulnerable to violations of the anonymity of thoir responses. 
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Building on this premise , Andreason (1970) hypothesized that for a mail 
questionnaire study of New York State Lottery winners, "The greater the 
impersonality of the correspondence accompanying the mail questionnaire, the 
greater the rate of return of delivered questionnaires" (p. 275). He 
operationally defined three levels of personalization: the least personal 
form used a general-purpose mimeographed salutation ("Dear Lottery Winner")? 
the next level of personalization was to address the individual through a 
handtyped salutation? the most personalized addressed the respondent indi- 
vidually and used a handwritten poscscript enjoining the person to respond 
with what Champion and Sear (1969, p. 337) ha e characterized as an egoistic 
appeal. In addition, for the follow-up reminder letters that were sent cut, 
one half were entirely mimeographed including the signature, and one half 
were entirely handwritten and hands igned. Andreason found no statistically 
significant differences in the return rates resulting from these various 
treatments. He concludes that "... the study supports Simon's finding 
that personalization is a variable with low potency for affecting response 
rate" (p. 277). However, this conclusion notwithstanding, he still main- 
tained that "... in some studies and for some respondents the expenditure 
will provide effects opposite to those desired" - thct is, return rates may 
be lower. 

Pre-Coding Personal Information 

In e good many mail questionnaire studies, it is often desirable, if 
not essential, to have some background data on the respondents so that a 
more detailed analysis of the responses may be undertaken. Typical back- 
ground data pertain to geographic location of the respondent, age, sex, 
socioeconomic status, marital status, and so on. In some instances, re- 
searchers may have such data on hand for the sample of interest prior to 
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sending out the questionnaires. In such cases, there are many advantages to 
pre-coding the respective background data on an individual on the question- 
naire sent to him or her. For example: (1) response burden is lessened 
because it 1 s not necessary to ask potential respondents for this informa- 
tion? (2) the accuracy and "believability" of such information is enhanced 
because the data sets will be complete and there will be no confounding 
effect due to individual interpretation of questions nor to variations in 
individuals 1 abilities to recall the information requested; (3) prior 
knowledge of such information facilitates a categorization of data that is 
more meaningful and better reflects the range of responses? (4) the adminis- 
trative burden associated with accommodating such data through pre-coding is 
appreciably less than the burden of similarly handling it after the ques- 
tionnaires are returned? and (5) the ethical issues of anonymity and con- 
fidentiality must be dealt with directly and in an "up-front" manner. 

The question does arise, however, as to what effect the inclusion of 
pre-coded personal information cn a questionnaire could have on the way in 
which people may respond. Perhaps because this kind of situation does not 
arise often, there has been little of relevance written about it. There 
does seem to be a consensus (see, for example, Berdie and Anderson, 1974) 
that when pre-coded information is included on a questionnaire, respoadents 
should be told what it is, and they should be reassured that their responses 
will indeed remain anonymous if that guarantee has been offered to them. 

Champion and Sear (1969) included pre-coded personal information on the 
questionnaires they used in their "methodological analysis" of questionnaire 
response rate. However, th^y side-stepped the issue of how that might have 
affected response rate by recording the pre-coded data on each questionnaire 
using "a special evanescent ink solution." Consequently, respondents had no 

ERIC 7 



5 



reason to assume that their anonymity was more apparent than real. Such an 
approach, however, raises the particularly difficult issue of the ethics of 
misinforming, or at least not fully disclosing to human participants in a 
research study as to just what the nature of the implied "contract" between 
them and the researcher is. 

Methods 

A mail-out questionnaire survey of 196 students dropping out of courses 
offered by a distance education university provided the opportunity to 
conduct the study reported here. The survey was an attempt to determine why 
the students had dropped out of the home study course they had enrolled in. 

In this study, the cover letter accompanying the questionnaire was 
personalized by making it "friendly" as compared to making it "formal." 
Specifically, the two versions of the covering letter differed in the 
following ways: 

"Personalized" Version "Formal" Version 



Indented style format. 
. Predominant use of active verbs. 
Predominant use of first and 
second person 

Informal, grammatically loose words 
and constructions (e.g. use of a 
postscript; short sentences? lower 
level of reading difficulty) . 
Appeal was more personal, direct 
and friendly. 

Letter was signed with full hand- 
written first and last name of 
researcher. 



Block style format. 

Predominant use of passive verbs. 

Predominant use of third person. 
More formal, grammatically 
precise words and constructions 
(e.g. use of conjunctives such as 
"moreover," "namely"). 
Appeal was more impersonal, 
indirect and pedantic. 
Letter was signed with handwrit- 
ten initials only of given names 
and full last name. 



Although stated in a form that differed according to these aforementioned 
dimensions, the content and directions given in the two types of letters 
were essentially the same. The letters were written by an experienced 
filter who was fully involved in the study and who understood the nature and 
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The University's student record system was used to identify the group 
of students dropping out and to generate mailing labels. The data in this 
system came from application forms filled in by students and consequently 
the records contain a substantial amount of background information on each 
student as well as information pertaining to courses taken and academic 
performance. Specific elements of interest were abstracted from the record 
of each student in the sample and were printed on the same kind of adhesive 
label that was used for the mailing information. Rather than ask students 
to provide this information again and risk incomplete and/or inaccurate 
responses, the labels were stuck onto the end of the questionnaire at the 
same time that the questionnaire was being prepared for mailing. This 
ensured that each individual received the appropriate questionnaire. One 
half the questionnaires simply explained what the coded data were for and 
reassured thp individual that these data would in no way compromise the 
anonymity of his/her response. The other half of the questionnaire similar- 
ly explained and reassured. However, in this case the label was mounted on 
a tear-away portion of the questionnaire and the student was informed that 
he or she could remove the coded information from the questionnaire if they 
were at all concerned that it would compromise their anonymity. It was 
presumed that providing subjects with this option would reinforce the effect 
of the assurance of anonymity. 

It may be worth emphasizing that responses were indeed anonymous (as 
opposed to being just confidential). The questionnaires were not numbered 
or labelled in any way such that any individual's responses could be 
identified. 

In all other respects, the questionnaires were identical, and the 
i.dM-icuals in the sample were treated in the same way. They all received 
identical reminder cards containing offset printed text and handwritten 



signature. All material was sent out with metered postage. A self- 
addressed envelope, also with metered postage, was enclosed with each 
questionnaire sent out. The reminder card was mailed to all individuals in 
the sample (since it couldn't be ascertained who had already responded and 
who hadn't) two weeks after the questionnaire was sent. As has been found 
in other studies, the mail-out of the reminder card was followed by an 
appreciable number of questionnaire returns. There was no significant 
difference in response to the reminder card among the treatment conditions. 

All respondents were offered a copy of the findings of the study partly 
as an indication of the sincerity of the researcher and partly to offer them 
some return for their effort. Requests were to have been made on a mail-in 
card that could be returned separate from the questionnaire. A very small 
number of students requested a copy of findings of the study. 

A Me a Culpa 

It should be acknowledged that a stronger test of whether the inclusion 
of pre-coded personal information affects response rate would have been to 
include a test group whose questionnaires did not include such information. 
However, it was felt that the value of the survey as an inquiry into why 
students discontinue university level home study would have been impaired 
had the study followed this design. Background data would have been avail- 
able for some students and not for others, and the investigator would have 
been faced with the considerable challenge of reconciling two sets of data - 
as well as having to account to the consumers of the study results why 
important data was available for some respondents but not others. 
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Data 

Students were randomly assigned to the two conditions of cover lette : 
and to the two conditions pertaining to the inclusion of pre-coded personal 
data, resulting in four combinations; personal letter and no option to 
remove pre-coded personal data; personal letter and option to remove pre- 
coded personal data; formal letter and no option to remove personal pre- 
coded data; and formal letter and option to remove personal pre-coded data. 
There were 49 subjects in each of these 4 possible treatment conditions. 

There were 77 responses in total, which translates to a return rate of 
39.3 percent. The returns were distributed as indicated in Table I. 

TABLE I 

DISTRIBUTION OF QUESTIONNAIRE RESPONSES 
BY TREATMENT CONDITION 

PERSONALIZED FORMAL 

LETTER LETTER SUBTOTALS 



Option to remove 

personal information 21 14 35 

No option to remove 

personal information 18 24 42 

Subtotals 39 38 N=77 

The return rate of 39 # 3 percent is almost certainly artificially 
depressed. Studies of ?5tudent dropout from university correspondence study 
indicate that appreciable numbers of students "disappear" shortly after 
enrolling and can't be contacted. Presumably questionnaires mailed to these 
people would similarly fail to make contact. Consequently, the nurtoer of 
deliverable questionnaires could have been substantially fewer than 196. 
However, in the absence of any hard evidence as to what this number really 
is, the full sample was taken to be 196. It was further assumed that the 
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number of questionnaires failing "to make contact" in each of the four 
treatment groups would be the same - thereby randomly distributing the 
effect of this spurious condition such that it could be assumed not to 
seriously affect inter-treatment group comparison. 

Results and Conclusions 

Previous studies by Linsky (1965) and Childers and Ferrell (1979) have 
demonstrated the value of using factorial designs to study factors thought 
to affect response rates . The often contradictory results obtained in 
studies of questionnaire methodology strongly suggest that various factors 
may work in combination to produce effects that would otherwise be absent if 
the factors were studies in isolation from one another. The studies con- 
ducted by Simon (1967) and Andreason (1970) are good examples of this. 

In the present study, it seemed reasonable to assume that if students 
were embarrassed about dropping out of their home study courses, or if they 
felt themselves to be failures, or if they were highly critical of the 
University, then they might feel particularly threatened if it appeared that 
the anonymity of their responses might be compromised. If this were so, 
then like Simon's and Andreason 1 s studies, the personal style of cover 
letter and the inclusion of pre-coded information with no option for remov- 
ing it should be associated with a lower return rate. It was also possible, 
of course, that the individuals receivina questionnaires with the tear-off 
option would choose to exercise that option but would still return a 
completed questionnaire. In the event, only one person out of the 35 
respondents in that treatment condition group chose to do so. 

A chi-squared test was applied to the 2x2 contingency table given 
above. The chi-squared test in this context was intended to be a test of 
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the degree of association between the two factors. A significant result 
would be taken as evidence of an interactive effect between the factors. A 
nonsignificant result would indicate that the factors were independent , and 
consequently interpreted as indicating no interaction effect. The obtained 
chi-squared value in this test was 1.608 , which is not significant at the 
0.05 level. 

Because the hypothesis of an interactive effect may be rejected, the 
question arises as to whether the main effects are significant. Chi~squared 
tests of the appropriate contingency tables (Tables II and III) indicate 
that neither of the two main effects was significant. 





TABLE II 






CHI- 


SQUARE TEST FOR EFFECT 


OF LETTER 


TYPE 




PERSONALIZED 


FORMAL 






LFTTER 


LETTER 


SUBTOTALS 


Response 


39 


38 


77 


No response 


59 


60 


119 


Subtotals 


98 


98 


196 



Calculated ty 1 = .021 Critical V (D=l, a«.05) - 3.84 



TABLE 

CHI-SQUARE TEST FOR 
OR NO OPTION TO REMOVE 



III 

EFFECT OF OPTION 
PERSONAL INFORMATION 



OPTION NO OPTION 

TO REMOVE TO REMOVE 

PERSONAL PERSONAL 

INFORMATION INFORMA HON SUBTOTALS 



Response 


35 


42 


77 


No response 


63 


56 


119 


Subtotals 


98 


98 


196 



Calculated ij> 2 = 1.048 Critical ij> 2 (D=l, a=.05) = 3.84 
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With respect to return rate , the results of this study indicate: 

1. There was no interactive effect between the kind of cover letter 
sent and whether or not subjects were given the option of removing 
the pre-coded personal data. Subjects were equally likely to 
respond whatever the combination of cover letter and whether or 
not they could remove the pre-coded data. 

2. Of those who responded , subjects receiving questionnaires with the 
option to tear off the pre-coded personal information did not 
exercise that option. 

3. There is no difference in the response rate according to whether 
an individual received a friendly cover letter or a formal cover 
lettei . 

4. There is no difference in the response rate according to whether 
an individual was offered or was not offered the option of remov- 
ing the pre-coded personal information on the questionnaire. 

Discussion 

The results from this study are congruent with the results reported by 
Simon (1967) and Andreason (1970). However, unlike these two studies, there 
is not even a "suggestive" indication in the present study that a person- 
alized letter is more effective in eliciting a response than is a formal 
letter. 

The finding that including pre-coded personal data on a questionnaire 
does not ir.pair return rates may be welcome news to investigators conducting 
studies in which they have prior knowledge of respondent characteristics. 
As mentioned previously, such a procedure facilitates the aggregation and 
processing of essential background information, while at the same time 
ensuring that it is ejcurate. Moreover, affixing such information before 
the questionnaire is sent to an individual - rather than afterward - has 
many advantages. Adding such information after the questionnaire has been 
sent out, requires that each questionnaire be uniquely identified. This 
means that extra bookkeeping would be required to ensure that a particular 
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individual's questionnaire is properly matched with the background data for 
that person. Moreover, there would be the problem of how to reconcile to 
the respondent the apparent discrepancy between identifying his/her response 
and the researcher's promise of confidentiality/anonymity. The approach 
described in this paper avoids these complications altogether. 

Finally, as indicated earlier, the return rate was only 39.3 percent. 
Unfortunately, some of the same motivational factors that lead to students' 
discontinuing distance study may also disincline them not to respond to a 
survey. The low return is particularly problematic with respect to the 
objective of the survey - namely, to attempt to find out why students were 
dropping out of courses. However, the study reported here pertains simply 
to the experimental conditions under which the study was conducted - with no 
direct reference to the substance of the survey. There is no question but 
that a higher response rate is always better, no matter which point of view 
is taken. However, it is not clear how serious a matter a lower response 
rate is in respect of testing the effects of various treatment conditions on 
rates of return. If one assumes that the "non-response" effect is randomly 
distributed across treatment conditions, then a low overall response rate 
(within reasonable limits) may not be so serious. However, if one has 
grounds for arguing that there could be an interactive effect between 
certain of the treatment etfects and non-response, then there could clearly 
be a problem. This is generally a problem with surveys in which the return 
rates are "low" - and not just a problem particular to the study reported 
here. 
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